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METHOD AND APPARATUS FOR AUTOMATICALLY SELECTING AN ALTERNATE 

ITEM BASED ON USER BEHAVIOR 

5 

Field of the Invention 

The present invention relates to recommendation 
systems , such as recommenders for television programming or other 
10 content, and more particularly, to a method and apparatus for 
automatically selecting an alternate recommended program or item. 

Background of the Invention 

The number of media options available to individuals is 
15 increasing at an exponential pace. As the number of channels 
available to television viewers has increased, for example, along 
l| : with the diversity of the programming content available on such 
^ channels, it has become increasingly challenging for television 
£ viewers to identify television programs of interest. 
jfO Historically, television viewers identified television programs 
of interest by analyzing printed television program guides. 
%j Typically, such printed television program guides contained grids 
f|f listing the available television programs by time and date, 
:;| channel and title. As the number of television programs has 
|45 increased, it has become increasingly difficult to effectively 
identify desirable television programs using such printed guides. 

More recently, television program guides have become 
available in an electronic format, often referred to as 
electronic program guides (EPGs) . Like printed television 
30 program guides, EPGs contain grids listing the available 
television programs by time and date, channel and title. Some 
EPGs, however, allow television viewers to sort or search the 
available television programs in accordance with personalized 
preferences. In addition, EPGs allow for on-screen presentation 
35 of the available television programs. 
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Many viewers have a particular preference towards, or 
bias against, certain categories of programming, such as action- 
based programs or sports programming. A number of tools are 
available that recommend television programming by applying such 
5 viewer preferences to the EPG to obtain a set of recommended 
programs. While such television program recommenders identify 
programs that are likely of interest to a given viewer, they are 
not foolproof, and often recommend programs that are not of 
sufficient interest to the viewer. Thus, the viewer must 
10 affirmatively interact with the television, set-top terminal or 
remote control to select an alternate program. 

A need therefore exists for a method and apparatus for 
automatically selecting an alternate program selection when a 
p viewer does not sufficiently like a current program selection. A 
!;|5 further need exists for a method and apparatus for evaluating the 
H reaction of a viewer to presented content in real-time and for 
x % selecting an alternate program when the viewer dislikes the 
4i currently selected content. Yet another need exists for a method 
and apparatus for automatically selecting an alternate program 
iJO without requiring a manual entry using a specific device. 

fit Summary of the Invention 

f:^ Generally, a method and apparatus are disclosed for 

automatically selecting an alternate item based on user behavior. 
25 The illustrative television programming recommender monitors 
viewer behavior and automatically selects an alternate program 
when the viewer does not sufficiently like the current program 
selection . 

One or more audio/visual capture devices are focused on 
30 the user to monitor user behavior and detect predefined negative 
behavior suggesting that the user does not like a currently 
selected program. The detected predefined negative behavior may 
include, for example, (i) auditory commands, (ii) gestural 



-2- 



US010107 



commands, (iii) facial expressions, or (iv) other predefined 
behavior suggesting that the user dislikes the program. 

Once predefined negative behavior is identified, an 
alternate program is selected. The present invention provides a 
flexible mechanism for providing an alternate program selection, 
since the user is not required to use a remote control or set-top 
terminal as an input mechanism. 

A more complete understanding of the present invention, 
as well as further features and advantages of the present 
invention, will be obtained by reference to the following 
detailed description and drawings. 

Brief Description of the Drawings 

FIG. 1 illustrates a television programming recommender 
in accordance with the present invention; 

FIG. 2 illustrates a sample table from the program 
database of FIG. 1; 

FIG. 3A illustrates a sample table from a Bayesian 
implementation of the viewer profile of FIG. 1; 

FIG. 3B illustrates a sample table from a viewing 
history used by a decision tree (DT) recommender; 

FIG. 3C illustrates a sample table from a viewer 
profile generated by a decision tree (DT) recommender from the 
viewing history of FIG. 3B; and 

FIG. 4 is a flow chart describing an exemplary 
alternate program selection process embodying principles of the 
present invention. 

Detailed Description 

FIG. 1 illustrates a television programming recommender 
100 in accordance with the present invention. As shown in FIG. 
1, the television programming recommender 100 evaluates each of 
the programs in an electronic programming guide (EPG) 130 to 
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identify programs of interest to one or more viewer (s) 140. The 
set of recommended programs can be presented to the viewer 140 
using a set-top terminal/television 160, for example, using well 
known on-screen presentation techniques. While the present 
5 invention is illustrated herein in the context of television 
programming recommendations, the present invention can be applied 
to any automatically generated recommendations that are based on 
an evaluation of user behavior, such as a viewing history or a 
purchase history. 

10 According to one feature of the present invention, the 

television programming recommender 100 monitors viewer behavior 
and automatically selects an alternate program when the viewer 
does not sufficiently like the current program selection. As 
0 shown in FIG. 1, the television programming recommender 100 
includes one or more audio/visual capture devices 150-1 through 
1^ 150-N (hereinafter, collectively referred to as audio/visual 
capture devices 150) that are focused on the viewer 140. The 
j# audio/visual capture devices 150 may include, for example, a pan- 
^ tilt-zoom (PTZ) camera for capturing video information or an 
||0 array of microphones for capturing audio information, or both. 
% The audio or video images (or both) generated by the 

flf audio/visual capture devices 150 are processed by the television 
programming recommender 100, in a manner discussed below in 
conjunction with FIG. 4, to identify one or more predefined (i) 
25 auditory commands, (ii) gestural commands, such as a " thumbs 
down," (iii) facial expressions, such as a sad or unhappy 
expression, (iv) other predefined behavior suggesting that the 
viewer dislikes the program, such as booing, walking away or not 
paying attention, or (v) a combination of the foregoing, 
30 hereinafter, collectively referred to as "predefined negative 
behavior . " 

Once predefined negative behavior is identified, the 
television programming recommender 100 can select an alternate 
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program and optionally update one or more viewer profiles 300, 
discussed below in conjunction with FIGS. 3A and 3C, in 
accordance with teachings of United States Patent Application 
Serial Number 09/718,261, filed November 22, 2000, entitled 
5 "Method and Apparatus for Obtaining Auditory and Gestural 
Feedback in a Recommendation System," assigned to the assignee of 
the present invention and incorporated by reference herein. The 
viewer behavior can be (i) explicit, such as predefined auditory 
or gestural commands; or (ii) implicit, such as information that 
10 may be derived from user behavior (or both) . In this manner, the 
present invention provides a flexible mechanism for providing an 
alternate program selection, since the user is not constrained to 
using a remote control or set-top terminal as an input mechanism. 
Q in a further variation, the present invention can 

:| 5 detect a change in the mood of a user and make an alternate 
H program recommendation based on the new mood of the user. For a 
^j; detailed discussion of a mood-based recommendation system, see 
United States Patent Application Serial Number 09/718,260, filed 
^ November 22, 2000, entitled "Method and Apparatus for Generating 
!|0 Recommendations Based on Current Mood of User," assigned to the 
J|jj assignee of the present invention and incorporated by reference 
Ill herein. 

!-t As shown in FIG. 1, the television programming 

recommender 100 contains a program database 200, one or more 

25 viewer profiles 300, and an auditory and gestural feedback 
analysis process 400, each discussed further below in conjunction 
with FIGS. 2 through 4, respectively. Generally, the program 
database 200 records information for each program that is 
available in a given time interval. One illustrative viewer 

30 profile 300, shown in FIG. 3A, is an explicit viewer profile that 
is typically generated from a viewer survey that provides a 
rating for each program feature, for example, on a numerical 
scale that is mapped to various levels of interest between 
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"hates' 7 and "loves," indicating whether or not a given viewer 
watched each program feature. Another exemplary viewer profile 
300' , shown in FIG. 3C, is generated by a decision tree 
recommender, based on an exemplary viewing history 360, shown in 
5 FIG. 3B. The present invention permits the survey response 
information, if any, recorded in the viewer profile 300 to be 
supplemented with the detected auditory or gestural feedback 
information. 

The alternate program selection process 400 analyzes 
10 the audio or video images (or both) generated by the audio/visual 
capture devices 150 to identify predefined negative behavior. 
Once such predefined negative behavior is identified, the 
alternate program selection process 400 automatically selects an 
Q alternate program, such as the program with the next highest 
^fc recommendation score. 

H The television program recommender 100 may be embodied 

as any computing device, such as a personal computer or 

4; workstation, that contains a processor 120, such as a central 

^' processing unit (CPU), and memory 110, such as RAM and/or ROM. 

||0 The television program recommender 100 may also be embodied as an 
application specific integrated circuit (ASIC), for example, in a 

t§ set-top terminal or display 160. In addition, the television 

g programming recommender 100 may be embodied as any available 
television program recommender, such as the Tivo™ system, 

25 commercially available from Tivo, Inc., of Sunnyvale, California, 
or the television program recommenders described in United States 
Patent Application Serial No. 09/466,406, filed December 17, 
1999, entitled "Method and Apparatus for Recommending Television 
Programming Using Decision Trees," (Attorney Docket No. 700772), 

30 United States Patent Application Serial No. 09/498,271, filed 
Feb. 4, 2000, entitled "Bayesian TV Show Recommender, " (Attorney 
Docket No. 700690) and United States Patent Application Serial 
No. 09/627,139, filed July 27, 2000, entitled "Three-Way Media 
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Recommendation Method and System," (Attorney Docket No. 700913), 
or any combination thereof, as modified herein to carry out the 
features and functions of the present invention. 

FIG. 2 is a sample table from the program database 200 
5 of FIG. 1 that records information for each program that is 
available in a given time interval. As shown in FIG. 2, the 
program database 200 contains a plurality of records, such as 
records 205 through 220, each associated with a given program. 
For each program, the program database 200 indicates the 
10 date/time and channel associated with the program in fields 240 
and 245, respectively. In addition, the title, genre and actors 
for each program are identified in fields 250, 255 and 270, 
respectively. Additional well-known features (not shown) , such 
q as duration, and description of the program, can also be included 
;|;5 in the program database 200. 

FIG. 3A is a table illustrating an exemplary explicit 
% viewer profile 300 that may be utilized by a Bayesian television 
recommender. As shown in FIG. 3A, the explicit viewer profile 
300 contains a plurality of records 305-313 each associated with 
Q|0 a different program feature. In addition, for each feature set 
Si forth in column 340, the viewer profile 300 provides a numerical 
ffii representation in column 350, indicating the relative level of 
2 interest of the viewer in the corresponding feature. As 
discussed below, in the illustrative explicit viewer profile 300 
25 set forth in FIG. 3A, a numerical scale between 1 ("hate") and 7 
("love") is utilized. For example, the explicit viewer profile 
300 set forth in FIG. 3A has numerical representations indicating 
that the user particularly enjoys programming on the Sports 
channel, as well as late afternoon programming. 
30 In an exemplary embodiment, the numerical 

representation in the explicit viewer profile 300 includes an 
intensity scale such as: 



Number 


Description 


1 


Hates 
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2 


Dislikes 


O 


Moderately negative 


4 


Neutral 


5 


Moderately positive 


6 


Likes 


7 


Loves 



FIG. 3B is a table illustrating an exemplary viewing 
history 360 that is maintained by a decision tree television 
recommender. As shown in FIG. 3B, the viewing history 360 
contains a plurality of records 361-369 each associated with a 
different program. In addition, for each program, the viewing 
history 360 identifies various program features in fields 370- 
379. The values set forth in fields 370-379 may be typically 
obtained from the electronic program guide 130. It is noted that 
if the electronic program guide 130 does not specify a given 
feature for a given program, the value is specified in the 
viewing history 360 using a 

FIG. 3C is a table illustrating an exemplary viewer 
profile 300 1 that may be generated by a decision tree television 
recommender from the viewing history 360 set forth in FIG. 3B. 
As shown in FIG. 3C, the decision tree viewer profile 300' 
contains a plurality of records 381-384 each associated with a 
different rule specifying viewer preferences. In addition, for 
each rule identified in column 390, the viewer profile 300' 
identifies the conditions associated with the rule in field 391 
and the corresponding recommendation in field 392. 

For a more detailed discussion of the generating of 
viewer profiles in a decision tree recommendation system, see, 
for example, United States Patent Application Serial No. 
09/466,406, filed December 17, 1999, entitled "Method and 
Apparatus for Recommending Television Programming Using Decision 
Trees," (Attorney Docket No. 700772), incorporated by reference 
above . 
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FIG. 4 is a flow chart describing an exemplary 
alternate program selection process 400. In the exemplary 
implementation of FIG . 4, the alternate program selection process 
400 monitors the user behavior during step 410. A test is 
performed during step 420 to determine if any predefined negative 
behavior is detected. If it is determined during step 420 that 
predefined negative behavior is not detected, then program 
control returns to step 410 to continue monitoring. 

If, however, it is determined during step 420 that 
predefined negative behavior is detected, then a further test is 
performed during step 430 to determine if the detected predefined 
negative behavior satisfies any additional specified heuristics 
or thresholds, such as a at least minimum amount of time 
remaining until the next program change. In other words, if 
there is only a relatively short amount of time remaining in the 
current selected program, then the predefined negative behavior 
will be ignored. Thus, if it is determined during step 430 that 
the detected predefined negative behavior fails to satisfy any 
additional specified heuristics or thresholds, then the 
predefined negative behavior is ignored during step 440. 

If, however, it is determined during step 430 that the 
detected predefined negative behavior satisfies any additional 
specified heuristics or thresholds, then program control proceeds 
to step 450, where a new program is selected. For example, the 
alternate program selection process 400 can optionally select the 
program with the next highest recommendation score. As 
previously indicated, can detect a change in the mood of a user 
and make an alternate program recommendation based on the new 
mood of the user, as described in United States Patent 
Application Serial Number 09/718,260, filed November 22, 2000, 
entitled "Method and Apparatus for Generating Recommendations 
Based on Current Mood of User," assigned to the assignee of the 
present invention and incorporated by reference herein. For 
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example, if the user is tired, a less intensive program may be 
selected, such as an action-based program over a drama. 

It is to be understood that the embodiments and 
variations shown and described herein are merely illustrative of 
the principles of this invention and that various modifications 
may be implemented by those skilled in the art without departing 
from the scope and spirit of the invention. 



